# Pre-trained language models

A legal domain-specific model based on the Italian XXL BERT model, with additional pre-training on 3.7GB of preprocessed text from the National Judicial Archive

ViHealthBERT is a pre-trained language model for Vietnamese health text mining, providing strong baseline performance in the healthcare domain

An Arabic Wikipedia Q&A system based on AraELECTRA, specifically designed for Arabic reading comprehension tasks

KoGPT2 is a Korean generative pre-trained model based on the Huggingface Transformers framework, developed and open-sourced by SKT-AI.

CDLM is a pre-trained model focused on cross-document language modeling, capable of processing semantic relationships between multiple documents.

A multi-dialect BERT model initialized with Arabic-BERT and trained on 10 million Arabic tweets, supporting identification of various Arabic dialects